Examining the Psychometrics of the National HIV Behavioral Surveillance measure for community HIV-related Stigma

The research tested the psychometrics of the Centers for Disease Control and Prevention’s National HIV Behavioral Surveillance(NHBS) community HIV-related stigma scale. Data was from men who have sex with men(MSM) NHBS cycles conducted 2011-2017 in Miami-Dade, Florida among N=1,455 participants. MSM were cis-gender male, 18+ years old, reported lifetime oral/anal sex with a male, and lived in Miami-Dade County. We assessed reliability using Cronbach’s alpha and McDonald’s omega, determined factors using principal factor analysis, and assessed construct validity using five a priori hypotheses. The scale was unidimensional, had questionable internal reliability (α=0.68, ω=0.69), and met four of five a priori hypotheses in the expected direction. Correlations were medium-weak in strength and only one was consistently met. Future iterations of the NHBS survey should consider replacing the 4-item community HIV-related stigma scale with an instrument that has superior internal reliability, measures multiple HIV-related stigma dimensions, and demonstrates stronger evidence of validity.


INTRODUCTION
Men who have sex with men (MSM) continue to face a disproportionate burden of new HIV diagnoses in the United States (U.S.), where in 2018 MSM comprised 66% of all new HIV diagnoses. (1) Although between 2014-2018, MSM experienced an overall 7% decrease in HIV incidence, there were large racial disparities present within this rate change; where white MSM experienced a 16% decrease, but rates among Black and Hispanic MSM remained stable. (1) In light of the disproportionate burden of HIV among MSM, it is of increased importance to study factors related to HIV outcomes in this population.
Beginning in 2003, the Centers for Disease Control and Prevention (CDC) created the National HIV Behavioral Surveillance (NHBS) survey to conduct behavioral surveillance among persons with increased risk of HIV acquisition. (2) The survey measures behavioral risk factors (e.g. sexual behaviors, substance use), HIV testing and prevention behaviors, and other community perception variables like HIV-and gay-related stigma. The NHBS provides an important tool for systematic monitoring of the impact of the National HIV/ AIDS strategy as well as providing a behavioral context for trends in HIV incidence. (2) Despite the NHBS's importance, little has been done to assess the reliability and validity of key measurements within the survey, namely HIV-and gay-related stigma.
HIV-related stigma is significantly deleterious to HIV-related health outcomes. (3)(4)(5)(6) As described by Turan et al. (2017), HIV-related stigma is comprised of 4 sub-dimensions including enacted, internalized, anticipated, and community stigma. (7) Enacted stigma can be described as discrimination based on one's HIV status, while anticipated stigma is the fear of the consequences after revealing one's HIV status. (7) Internalized stigma are negative feelings about oneself due to one's HIV status, while community stigma is the perceived negative feelings of people living with HIV (PLWH) within one's community. (7) Of the 4 sub-constructs comprising HIV-related stigma, the NHBS survey only measures community HIV-related stigma. Previous research using NHBS data has found significant association between community HIV-related stigma and decreased age, (8) non-white race/ ethnicity, (8,9) HIV status miss-reporting, (10) antiretroviral therapy (ART) non-adherence, (11) and decreased preexposure prophylaxis (PrEP) awareness/willingness. (12) However, the measurement of community HIV-related stigma was inconsistent between studies, where some studies used single items, converted the ordinal item into a binary variable, and/or utilized a mean item score. Consistency in measurement operationalization is important as it allows for better comparability of results between NHBS sites. Despite the deleterious effect of community HIV-related stigma, no research has examined the validity of the scale used in the survey. Establishing validity and reliability of a scale is important as it ensures researchers are measuring what they intend to measure and that they measure the same construct each time.
The current research was conducted to test the psychometrics of the NHBS community HIV-related stigma scale. We hypothesized the scale would be unidimensional given findings from previous studies on HIV stigma (13,14) where community HIV-related stigma items loaded onto their own factor. We anticipated that the scale would have good internal consistency that would be temporally consistent. Finally, we hypothesized that the community HIV-related stigma scale would have weak to moderate positive correlation with distress and a positive HIV test result, and would have a weak to moderate negative correlation with community gay-related tolerance, PrEP awareness, and ART adherence.

METHODS
The NHBS survey is an anonymous serial, cross-sectional survey currently conducted in 22 cities across the U.S. to assess the association of behavioral factors on HIV surveillance data. NHBS is conducted among three different key groups: men who have sex with men, people who inject drugs, and heterosexuals at increased risk for HIV. We used data from the CDC NHBS survey conducted in Miami-Dade County, Florida among MSM in 2011, 2014, and 2017 (MSM3, MSM4, MSM5, respectively). MSM were recruited using venue-based time-space sampling (VBS), described in previous studies. (15) To be eligible for participation, MSM had to identify as cis-gender male, 18+ years of age, report oral or anal sex with a male in their lifetime, able to complete the interview in English or Spanish, and report living in the Miami-Dade County, Florida. Study staff systematically approached and screened men to establish eligibility for participation. Eligible MSM consented and completed an interviewer-driven survey using a tablet computer. Questions assessed demographics, sexual and drug use behaviors, HIV prevention and testing behaviors, and perceived community stigma. All participants were offered an HIV test. Participants during MSM3 and MSM4 received $25 for completing the survey and $25 for completing the HIV test and $35 for completing the survey and $40 for completing the HIV test for MSM5.

Community HIV-related Stigma
Community HIV-related stigma was measured using a scale adapted from two previous studies. (13,14) The scale was comprised of 4 items with 5-point Likert style responses from "strongly disagree=1" to "strongly agree=5" (e.g. Most people in Miami think that people who got HIV through sex or drug use have gotten what they deserve.). The item, "Most people in Miami would support the rights of a person with HIV to live and work wherever they wanted to," was reverse coded to be in the same direction as the other items. Total possible scores could range from 4-20, where higher scores indicated greater perceived community HIV-related stigma.

Measurement of Construct Validity
Community gay-related tolerance-Community gay-related tolerance was measured with the item, "Most people in Miami are tolerant of gays and bisexuals," using 5-point Likert options from "strongly disagree=1" and "strongly agree=5." Distress-Psychological distress was measured using the Kessler Psychological Distress scale (K6) (α=0.86) (16) only in the MSM5 survey. The K6 scale contains 6 items with 5-point Likert style responses from "none of the time=0" to "all of the time=4" (e.g. During the past 30 days, how often did you feel hopeless?). Scores could range from 0-24 where higher scores indicated higher levels of distress.
HIV test result-All participants of MSM3-MSM5 were offered a rapid HIV test to be conducted at the end of the questionnaire. Participants were classified as testing positive (1) or negative (0). Those with an indeterminate test result were removed from the analyses.
PrEP Awareness-In the MSM3 survey, self-reported HIV-negative participants were asked the yes/no question, "Before today, have you ever heard of people who do not have HIV taking antiretroviral medicines, to keep from getting HIV?" In the MSM4 survey, self-reported HIV negative participants were asked the yes/no question, "Before today, have you ever heard of people who do not have HIV taking PrEP, the antiretroviral medicine taken every day for months or years to reduce the risk of getting HIV?" In the MSM5 survey, self-reported HIV-negative participants were asked the yes/no question, "Preexposure prophylaxis, or PrEP, is an antiretroviral medicine, such as Truvada, taken for months or years by a person who is HIV-negative to reduce the risk of getting HIV. Before today, have you ever heard of PrEP?" ART Use-In MSM3-MSM5 surveys, self-reported HIV positive participants were asked the yes/no question, "Are you currently taking antiretroviral medicines to treat your HIV infection?"

Analyses
All analyses were conducted in SAS Studio®. Frequencies and percentages were calculated to display pooled and stratified participant characteristics of MSM3-MSM5 cycles. After, principal factor analysis was conducted with promax rotation to determine if the community HIV-related stigma scale had any latent factors. An oblique rotation was used as we hypothesized that if more than one factor arose, they would be correlated. Scree test and eigenvalues of >1 and factor loading of >0.30 were used to determine factors and loadings respectively. (19) Internal reliability of the scale was assessed using Cronbach's alpha and McDonald's omega using the developed SAS macros by Hayes & Coutt (2020). (20) We include Cronbach's alpha to allow for comparison of our results to other studies, given its widespread use throughout the literature. However, use of Cronbach's alpha to assess reliability has some limitations outlined in previous literature, (20)(21)(22) particularly in scales with less than 10 items, (22) leading us to also include the preferred reliability measure of McDonald's omega. Pearson's and Spearman's correlation were conducted to determine correlations established in the a priori hypotheses, depending on whether the variable was continuous or ordinal, to determine construct validity using the proc corr statement in SAS.
We handled missing data using listwise deletion. α was set to 0.05.
The Cronbach's alpha for the pooled sample of the 4-item community HIV-related stigma measure was α=0.68, ranging from α=0.64-0.71 between cycles. The McDonald's omega for the pooled sample of the 4-item community HIV-related stigma measure was ω=0.69, ranging from ω=0.65-0.72 between cycles. None of the items were found to be deleterious to the internal reliability. Factor analysis indicated scale unidimensionality (pooled sample: Eigenvalue=1.29, Difference 1.36) and all items had factor loadings above 0.40 (Table II).
Overall, the pooled sample reported low-moderate levels of community HIV-related stigma (Table II) with a total scale average of 11.51±2.99 and a range of 4-20. In the pooled sample, two of the four a priori hypotheses were met where community HIV-related stigma was negatively correlated with community gay-related tolerance (r s =−0.22; p<0.001) and PrEP awareness (r s =−0.07; p=0.024). In MSM3 and MSM4, one of four a priori hypotheses were met where community HIV-related stigma was negatively correlated with community gay-related tolerance (r s =−0.15; p<0.001, r s =−0.22; p<0.001, respectively). In MSM5, three of five a priori hypotheses were met, where community HIV-related stigma was positively correlated with distress (r=0.23; p<0.001) and negatively correlated with community gayrelated tolerance (r s =−0.29; p<0.001) and testing HIV positive (r s =−0.09; p=0.044). All met hypotheses were in the appropriate direction, with the exception of testing HIV positive as negatively associated with community HIV-related stigma. Based on Cohen's guidance for interpreting correlation coefficients, (23) the strength of the significant relationships was between small and medium (Table III).

DISCUSSION
This was the first study to test the psychometrics of the community HIV-related stigma scale used in the NHBS survey among MSM. The unidimensional scale had acceptable internal consistency, but only consistently reached one of four a priori hypotheses in establishing construct validity (not including distress from MSM5). Though at different points the scale was significantly correlated with constructs directly associated with HIV-related stigma (e.g. distress, community gay-related tolerance, HIV test result, and PrEP awareness) the correlations were weaker than expected, and it was not significantly correlated with manifestations of community HIV-related stigma, such as current ART use. The findings of this study may imply the need to use a new HIV-related stigma scale in the NHBS survey. The national NHBS sample may be large enough to find significant differences between community HIV-related stigma and HIV-related health outcomes; however, the use of the community HIV-related stigma scale to find local level differences, where sample sizes are smaller, may limit the ability to use tailored data to support Florida's plan to eliminate HIV transmission and reduce HIV-related deaths (24) and inform state leaders on potential targets for stigma reduction interventions among MSM. Moreover, as previous research suggests, (6,25,26) other dimensions of HIV-related stigma not measured in the NHBS survey (e.g. enacted, internalized, and anticipated) may have a larger effect size with HIV-related health outcomes.
Additionally, the reliability of the community HIV-related stigma scale was questionable, suggesting that other measures with superior internal reliability should be considered. One measure to consider is the Multiple Discrimination Scale (MDS) developed by Bogart et al. (2013) that measures multiple dimensions of stigma related to HIV, race/ethnicity, and sexuality all with good internal reliability (α>0.83). (27) Another measure to consider is the 12-item HIV stigma scale, (28) adapted from the 40-item Berger HIV stigma scale, (29) that measures internal, enacted, community, and anticipated stigma with good internal reliability (α>0.80). Based on the results, future iterations of the NHBS survey should consider replacing the 4-item community related stigma scale with an instrument that 1) has superior internal reliability, 2) measures multiple dimensions of HIV-related stigma, 3) shows stronger evidence of validity.
Overall, perceived community HIV-related stigma was low-moderate in the sample. The majority of the items had mean scores that fell between "disagree" and "neither agree nor disagree"; however, the item, "Most people in Miami would discriminate against someone with HIV," had a score that had a mean that fell between "neither agree nor disagree" and "agree." Future qualitative research should further examine whom participants considered "most people in Miami" as well as what specific types of discrimination would be enacted (i.e. name calling, gossip, violence, etc.).

Limitations
This study had a number of limitations. First, the sample was comprised only of MSM in Miami-Dade County, Florida, meaning that the findings may not be generalizable to MSM in other NHBS study sites. Future research utilizing the nationwide NHBS sample may provide stronger evidence of instrument psychometrics. Secondly, participants recruited by the VBS recruitment method may not be generalizable to MSM who do not attend the targeted venues. Thirdly, we were unable to examine correlations between the community HIV-related stigma scale and other HIV stigma measures, as the main study was not designed to validate the scale. Lastly, participants who were willing to participate in a study surrounding HIV behavior may have had lower perceptions of community HIV-related stigma.
Despite the limitations, our study had some key strengths. Firstly, the sample used to validate the community HIV-related stigma scale was diverse, with the majority of participants falling into racial/ethnic HIV risk categories. Additionally, examining reliability and validity by NHBS cycle allowed for examination of instrument stability through time.

CONCLUSIONS
We successfully tested the psychometrics of the NHBS survey community HIV-related stigma scale. Questionable internal reliability and poor correlation with HIV-related health outcomes suggest the potential necessity for a new scale to measure HIV-related stigma. If deemed necessary, the future scale should be an instrument that 1) has superior internal reliability, 2) measures multiple dimensions of HIV-related stigma, 3) shows stronger evidence of validity.

Funding:
The research was funded in part by a contract with the Florida Department of Health (PI: Algarin) and the National Institute on Drug Abuse (T32 DA023356).